Detecting Cognitive Appraisals from Facial Expressions for Interest Recognition

Interest makes one hold her attention on the object of interest. Automatic recognition of interest has numerous applications in human-computer interaction. In this paper, we study the facial expressions associated with interest and its underlying and closely related components, namely, coping potential, novelty and complexity. We develop a method for automatic recognition of visual interest in response to images and micro-videos. To this end, we conducted an experiment in which participants watched images and micro-videos while their frontal videos were recorded. After each item they selfreported their level of interest, coping potential and perceived novelty and complexity. We used OpenFace to track facial action units (AU) and studies the presence of AUs with interest and its related components. We tracked the facial landmarks and extracted features from each response. We trained random forest regression models to detect the level of interest, curiosity, and appraisals. We obtained promising results on coping potential, novelty and complexity detection. With this work, we demonstrate the feasibility of detecting cognitive appraisals from facial expressions which will open the door for appraisaldriven emotion recognition methods.